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Abstract. In this paper, we introduce an approach to overcome the low 
accuracy of the Content-Based Image Retrieval (CBIR) (when using the 
global features). To increase the accuracy, we use Harris-Laplace detector 
to identify the interest regions of image. Then, we build the Region-Based 
Image Retrieval (RBIR). For the efficient image storage and retrieval, we 
encode images into binary signatures. The binary signature of a image is 
created from its interest regions. Furthermore, this paper also provides an 
algorithm for image retrieval on S-tree by comparing the images’ signatures 
on a metric similarly to EMD (earth mover’s distance). Finally, we evaluate 
the created models on COREL’S images. 


1. Introduction 


It is difficult to find images in a large set of them. A solution is to label the 
images [I, 2] but it is costly, time-consuming and unfeasible for many applica¬ 
tions. Moreover, the labeling process depends on the semantic description of 
image. So, the image retrieval system which is based on content is developed to 
extract visual attributes for the description of image content [3]. Some digital 
image retrieval systems have built, such as QBIC, ADL, DBLP, Virage, Alta 
Vista, SIMPLIcity, etc. 
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In recent years, there are some researches regarding CBIR are published, 
such as image retrieval based on color histogram [1, 2], image retrieval on the 
base of binary bit string and S-tree [3], extracting objects on the image based 
on the change of histogram value [5], the similarity image retrieval based on 
the comparison of feature regions and their similarity relationship on image [6], 
color image retrieval based on the detection of local regions by Harris-Laplace 
method [7], color image retrieval based on bit plane and L*a*b* color space [8], 
changeable color space to query the content of color images [9], etc. 

The paper approaches the semantic description of image contents through 
binary signatures and stores them on a S-tree. The S-tree data structure de¬ 
scribes the relationship between binary signatures. So, it describes the relation¬ 
ship in the contents of images. On the base of the description of the semantic 
relationship of image contents of the S-tree data structure, the paper finds out 
the similarity images by the content on COREL images [12]. 

The paper builds the similarity image retrieval on the base of the local 
interest regions. First of all, it extracts the interest regions using Harris-Laplace 
method. Then, the paper creates interest regions for image. Basing on these 
ones, it creates binary signatures and evaluates the similarity of images. In 
order to speed up the query, the paper presents S-tree to store binary signatures 
so as to build a similarity image retrieval algorithm on the S-tree. The paper 
contributes two main sections that reduce the amount of storage space and 
speed up the query of image objects on a large image database. 

The binary signature of images is used to describe the image’s content. 
When compared to GCHs (Global Color Histograms) and CCVs(Color-Coherence 
Vectors), the binary signature method in VBA (Variable-Bin Allocation) saves 
over 75% and 87.5% in storage overhead, respectively. [3] 


2. Data structure and similarity measure 


2.1. Binary signature 


In accordance with [4], the binary signature is formed by hashing the data 
objects, and it has k bits 1 and (m — k) bits 0 in the bit chain [l..m], where m 
is the length of the binary signature. The data objects and the object of the 
query are encoded on the same algorithm. When the bits in the signature data 
object cover completely the ones in the query signature, the data object is a 
candidate for the query. There are three cases: (1) the data object matches 
the query: each bit in the Sq is covered with the ones in the signature Si of 
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the data object (i.e., Sq A Si = Sq); (2) the object does not match the query 
(i.e., Sq A Si ^ Sq ); (3) the signatures are compared and then given a false drop 
result. 


2.2. EMD distance 


Suppose that / is a set of suppliers, J is a set of consumers, Cij is the 
transportation cost from supplier t € / to consumer j € J, we need to find 

out flows fij to minimize the total cost ^ ^ with the constraints [10, 

iei j&J 

11]: /y — Oj X] fij — J/j) X] fij ^ € I,j € J, where Xi is the provider’s 

i&I j&J 

general ability i G I, yj is the total need of the consumer j G J. The feasible 

condition is X] J/j — X] ^i- The EMD distance [10, 11] equals EMD{x,y) = 
jeJ iGl 

XligJ E,j^J 'Ej^J 

Eji^lEjj^J fij Elj^jVj 

Each image in the database is quantized into a fixed number of n colors 
Cl, C 2 ,..., c„. Each color cj is represented by a bit string of length m, i.e., 
lo': ^ < j ^ n, so each image can be described as a sequence of bits 
S = b\bl...bl-bfblf...b^. 

The binary signature of image I is SIGj = where = 

lr{b 2 --.bf^, bl € {0,1}. The weight of Bj component equals Wj = w{Bj) = 

m 

EiK X — X 100). Therefore, we have a weight vector of image / as W/ = 

i—1 

{w},wj, Let J be the image which needs to calculate the similarity 

corresponding to the image I. We need to minimize the cost to convert color 

n n 

distribution ^ ^ dijfij, where F = (fij) is a color distribution flow matrix 
i=lj = l 

between c} and Cj. Let D = (dij) be a Euclidean distance matrix in RGB color 
space between c\ and Cj. 


We set Wni = min( w\, w^j) and Wm = max( w], ^ w^j). The 

i—1 ^=1 3—^ 

distance EMD(I, J) is the color distribution flows from an image with the color 
weight as Wm to an image with color weight as W^,. We have the similarity 
measure between two images I and J with formula as follows: EMD{I, J) = 

(E E d ijfij) n n n n . 

min -, with X) X) f,j = W^ = min(^ w}, w^j) 

^ ^ fij i=l j — 1 i=l j — 1 
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2.3. S-tree 


S-tree [3, 4] is a tree with many balanced branches. Each node of the S-tree 
contains a number of pairs {sig, next), where sig is a binary signature and next 
is a pointer to a child node. The root of S-tree contains at least two pairs and 
at most M pairs {sig, next). All internal nodes in the S-tree can accommodate 
at least m and at most M pairs {sig, next) , 1 < m < M/2. The leaves of the 
S-tree contain image’s binary signatures sig, along with unique identification 
aid for those images. The S-tree most height for n signatures is h — log^n — 1. 

Building the S-tree is done by the inserting and splitting. The S-tree only 
contains a null leaf at the beginning. Then, we insert signatures into the S-tree. 
When the node v is full, we split into two nodes. At the same time, the parent 
node Vparent Is Created (if not exist). Simultaneously, two new signatures are 
inserted into node Vparent- 




physical page of node 


Figure 1. A sample of S-tree 
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Figure 2. A sample of nodes and leaves in S-tree 

The algorithm creates the S-tree that stores the image’s binary signature 
based on the EMD distance as follows: 
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Algorithml. Create the S-tree from the set of signatures 
Input: S = {{sigi,oidi)\i = 

Output: The Stree. 

1: Step 1. 

2: V = Root; 

3: if S' = 0 then STOP; 

4: else 

5: Choosing {sig,oid) G S; S = S\{sig,oid); 

6: Go to Step 2. 

7: end if 
8: Step 2. 

9: if V is leaf then 
10: V = vU {sig, oid); 

11: UnionSignature(u); 

12: if V. count > M then 

13: SplitNode(u); 

14: To go back Step 1; 

15: end if 

16: else 

17: EMD{SIGo —t sig, sig) = Ta\n{EMD{SIGi -G sig,sig)\SIGi G n}; 

18: V = SIGq -G next; 

19: To go back Step 2; 

20: end if 

21: return Stree 

Procedurel. Union of binary signatures at node v 
Input: Node v 

Output: The S — tree after union of signatures 
1: procedure UNIONSlGNATURE(Upareni) 

2: s = y sig^, with sig^ G v; 

3: if Vparent- = null then 

4: SIGy = {SIGi\SIGi -G next = V, SIGt G Vparent}; 

5: Uparent ^ {SIGy ^ ~ 

6: UnionSignature(vparent); 

7: end if 

8: end procedure 

The Algorithml creates the S-tree from the signature file S. Each signature 
sig G S is inserted into the most appropriate leaf. If the leaf is full, it would 
be split. Then, the S-tree grows in the direction of the root. 

The S-tree has many balanced branches. With each node of the S-tree 
is traversed in the best similarity measure direction with EMD distance The 
height of S-tree is h = [log^n — 1]. So, the cost of the query process on S- 
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tree is k x M x [log^n — 1], where k is the length of each signature, m is the 
minimum number of signatures, M is the maximum number of signatures of 
a node in the S-tree. However, if the appropriate leaf is full, it would be split 
based on a — seed, /? — seed operations ([4]) and the similarity measure EMD, 
which is done as follows: 

Procedure2. Split node v 
Input: a node v 

Output: the Stree (after splitting the node v) 

1: procedure SplitNode(v) 

2: Create the nodes Va and vp contain a — seed and (3 — seed] 

3: for SIGi € V do 

4: if EMD{SIGi —>■ sig,f3 — seed) < EMD{SIGi —>■ sig,a — seed) 

then 

5: Va = Vai-> SIGi] 

6: else 

7: Vp = VpiJ SIGi] 

8: end if 

9: end for 

10: Sa = U sigf, with sigf e Va] 

11: S/S = U G vp] 

12: Vparent — '^parent C Sq,, 

13: Vparent — Vparent C S^, 

14: if Vparent -count > M then 

15: SplitNode(upQ,-e,^i) , 

16: end if 

17: end procedure 


3. Image retrieval 


3.1. Extracting interest regions 


In order to extract the visual features of images, we standardize the image 
size (i.e. converting input image in different sizes into the image with the size 
of fc X fc). After that, we extract the color feature of images. Because the image 
based on JPEG standard is described on the YCbCr color space, we need to 
use YCbCr to extract the features of the images. 

According to [7, 8], the Gaussian transformation by human’s visual system 
is given by L{x, y) = ^[6.G{x, y, 6d)*Y +2.G(x, y, fe)*C6+2.G(x, y, fe)*Gr], 
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Figure 3. A sample of interest regions 


I 2_i_ 2 

with G{x,y, Sd) = g )• intensity Io{x,y) for color image 

is calculated with equation Io{x,y) = Det{M{x,y)) — a.Tr‘^{M{x,y)), where 
Det{u),Tr{u) are Determinant and Trace of matrix, M{x,y) is a second mo¬ 
ment matrix which can be defined as-M(a;, y) = djg,.G(x,y,Sj)* 

where Si,Sd are the integration scale and differentiation scale, and Lq, is the 
derivative computing the a direction. The interest points of color image are 
extracted from formula Iig(x,y) > Iq{x', y') with x',y' G A and lQ{x,y) > 0, 
where A is the neighborhood of point (x, y) and 0 is a threshold value. Let 
Oi = {o}, of,..., o”} be a set of interest circles, with the centers as the interest 
points and the set of interest radius as Rj = ...,r”}. The values of in¬ 

terest radius are in [0, min(M, N)/2], where M, N are the height and the width 
of image. They are extracted by LoG method (Laplace-of-Gaussian). For each 
image, the process of extracting interest points is described as follows: 
Algorithm2. Extract region-of-interest in image 
Input: Image /, threshold 9, scales 5i,5d 
O utput: Interest region of image O/ = {o}, of,..., o^} 

1: Step 1. For each pixel p G I, convert from RGB color space {R,G,B G 


[0,1]) to YGbGr color space as follows: 


Y 


65.481 

128.553 

24.996 


■ R ' 


16 

Cb 

= 

-37.797 

-74.203 

112 


G 

+ 

128 

Cr 


112 

-93.786 

-18.214 


B 


128 


3: Step 2. Perform Gaussian transform compiled with the human visual sys¬ 
tem to calculate the L{x,y) as follows: 

4: L(x, y) = ^ [6.G(x, y, Sd) *Y + 2.G(x, y,SD)*Cb + 2.G{x, y, Sd) * Gr] 

5: Step 3. Galculate the feature intensity Io{x,y) for color images as follows: 
6: Ioix,y) = Det{M{x,y)) - a.Tr'^{M{x,y)) 

7: Step 4- Gollect the set of interest points P as follows: 

S: P = {p{x, y) G I\Ioix, y) > loix', y') A /o(x, y) > 6», (x', y') G A} 

9: Step 5. Implement of the extraction interest regions O/ = {of, of,..., o”} 
based on the set of interest points P. 

10: return Oj = (of, of,..., o"} 


Px LxLy 

T T T ^ 

-P-^x-P-^y 
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3.2. Creating binary signature of an image 


With each interest region o] G 0i of an image I, the histogram is calcu¬ 
lated on the base of the standard color range C. Effective clustering method 
relied on Euclidean measure in RGB color space classifies colors of each pixel 
on the image. Let p be a pixel of image I so we have a color vector in RGB as 
Vp = {Rp, Gp, Bp). Let Vm = {Rm, Gm, Bm) be a color vector of the set of stan¬ 
dard color range C, so Vm = min {\\Vp — Vi\\,ViGG}. At that time, pixel p can 
be standardized in accordance with color vector Vm- According to experiment, 
the paper is used the standard color range on MPEG7 to calculate histogram 
for color images on COREL database. We let o\ G Oi {i = be the 

interest circles of the image I, the histogram vector of the circle o} is H{o\) = 

{Hi{o\),H 2 {o\), ...,Hn{o\)}. If we let hk{o\) = a standardized his- 

1 ^ 7 ) 

3 

togram vector will be/i(o}) = {/ii(o}),/i 2 (o}),..., The binary signature 

describes hk{o\) as Rj = b}b'j...bf' with 6^ = 1 if j = [(hj{o\) -1-0.05) x m], 
otherwise b^j = 0. So, the signature describes the interest region o} G Oi as 
Sig{ol) = B)Bj...Bf. For this reason, the binary signature of the image I is 
Si = Ufci Sig{o\). The process of creating binary signatures for color images 
is described as follows: 

Algorithms. Creating binary signature of image / 

Input: Image I, the look-up table of color vectors G = (Ei, V 2 , E„) 

Output: Binary signature Sj of image I 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 


Step 1. 

Oi = Algorithm2(/,6»,(5/,(5D); (i.e. O 7 = {o}, of,..., of}) 

Initialize binary signature Si = ..R°, with Bj = bib^-.-b'^, 6 ° = 0, 

i = 1,...,TO, j = l,...,n. 

Step 2. 

Initialize histogram vector H{o}) = {Hi{o}), ..., i7„(oj)} 
for pixel p € 0 } do 

Calculate color vector Vp = {Rp, Gp, Bp) 

Vm = min{\\Vp-V\\,V G C} 

Hm{0}) = Hm{0}) + 1 

end for 

Step 3. 

Initialize vector h{o}) = {hi{o }),..., hn{o\)} 
for hk{o\) G h{o\) do 


15: end for 

16: Step 4- 





RBIR using Interest Regions and Binary Signatures 


9 


17: for hk{o]) G h{o]) do 

18: for i=l..m do 

19: if j = [ihj{o]) + 0.05) X m] then 

20: 6 J = 1 

21: else 

22 : b^j = 0 

23: end if 

24: end for 

25: B’} = b]bj...bf 

26: end for 
27: Sig{oi) = 

28: Step 5. 

29: S'/ = S'/ V Sig(o\) 

30: Oi = Oi\{o}} 

31: ii Oi (/} then 
32: To go back Step 2. 

33: else 

34: return S/ 

35: end if 

The Algorithms creates a binary signature of image I . At Stepl basing on 
Algorithms, it extracts the interest regions O/ = At Step2, it 

calculate the histogram vector H{o\) = {Hi{o\),Hn{o\)} based on interest 
regions o} G O/ with the set of standard color C. At StepS, it standardize 
vector h{o}) = /i„(o})}. At Step4, it creates the binary signature 

for hk{o\) as B^ = b]b‘j...bY with 6^ = 1 if j = [{hj{o\) + 0.05) x m], otherwise 
bi’j = 0. At that time, the signature describes the interest region o} G 0/ as 
Sig{o{) = BjBj...BJ. At Step 5, it creates the binary signature of image I as 
Si = Uti Stg{o}). 

3.3. Image retrieval algorithm 


After storing the signatures and identifications of the corresponding images 
on the S-tree, the query process finds out the binary signatures of similarity 
images on the base of traversing S-tree. After finding the image’s signatures 
and relying on identifications of images, we can find out a set of similarity 
images corresponding to a query image. For this reason, the problem needs 
to find out the signatures of the images and the corresponding identifications. 
This query process is performed by proposed algorithm as follows: 
Algorithmd. Query image in S-tree 
Input: query signature sig and S — tree 
Output: Set of image signatures and identifications 
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1 : u = root; SIGOUT = 0; Stack = 0; 

2 : Push(Stack, v); 

3: while not Einpty(S'tacfc) do 
4: V = Pop(S'tocfc); 

5: if V is not Leaf then 

6 : for SIGi G V and SIGi —)■ sig A sig = sig do 

7: EMD{SIGo —>■ sig, sig) = inhi{EMD{SIGi —)■ sig, sig)\SIGi € 

v}; 

8: Push(Stack, SIGq —>■ next); 

9: end for 

10: else 

11 : SIGOUT = SIGOUT \J {{SIGi sig,oidf)\SIG^ € v}; 

12: end if 

13: end while 

14: return SIGOUT; 

Searching process is done similarly to traversing one the S-tree. So, the cost 
of the query process on the S-tree is kx M x [log^n — 1] where k is the length 
of each signature, m is the minimum number of signatures, M is the maximum 
number of signatures of a node in the S-tree. 


4. Experiments 


4.1. Image retrieval model 

The experimental process includes two phases. The first one performs pre¬ 
processing to convert image data into binary signatures and puts them into 
S-tree based on the similarity measure EMD. The second one performs query 
process corresponding to a query image which can be converted into a binary 
signature and queried on the S-tree on the base of similarity measure EMD. 
After finding the signatures of similarity images, we find out similarity images 
and arrange them with similarity measure EMD. 

Phase 1 : Perform pre-processing 

Step 1. Extract the features of images in the database to form feature 
vectors. 

Step 2. Convert feature vectors of images into binary signatures. 

Step 3. Calculate in turn the EMD distance of image’s signatures and insert 
them into the S-tree. 

Phase 2: Perform query 
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Figure 4. Model of region-based image retrieval system 


Step 1. With each query image, we extract the feature vector and convert 
it into binary signature. 

Step 2. To implement the binary signature retrieval process on S-tree in¬ 
cluding image’s signatures, we find out similarity images at each leaf node of 
tree through the similarity measure EMD. 

Step 3. After having similarity images, we arrange them in order of the 
similarity from high to low level and give the list of images. Then, we arrange 
them on the basis of similarity measure EMD. 


4.2. Experimental Results 


The experimental process is queried on COREL sample data [12] including 
10,800 images which can be divided into 80 different subjects. With each query 
image, we retrieve images on COREL data as so as find out the most similar 
ones to the query image. Then, we can compare to a list of the subjects of 
images to evaluate the accurate method. 
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Figure 5. Some results of image retrieval system 




Figure 6. (a)-number of operators to create S-tree; (b)-the time (milliseconds) 
to create S-tree 
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Figure 7. (a)-number of operators to query image; (b)-the time (milliseconds) 
to query image 




Figure 8. (a)-Recall; (b)-Precision 
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5. Conclusion 


The paper gives the evaluation method of similarity between two images on 
the base of binary signatures. At the same time, it simulates the experimental 
application on classification of image data of COREL. The experiment shows 
the evaluation method based on binary signatures and S-tree which can speed 
up querying similarity images as quickly as possible compared with the linear 
query. However, the use of feature color gives an inaccurate results with the 
meaning of image’s contents. Therefore, the next development extracts the 
feature objects on images. So, it builds a binary signature to describe objects 
as well as the content of images. At the same time, it creates a data structure to 
describe the relationship on the base of similarity measure between the images. 
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